Utopia: a Load Sharing Facility for Large, Heterogeneous Distributed Computer Systems

Load sharing in large heterogeneous distributed systems allows users to access vast amount of computing resources scattered around the system and may provide substantial performance improvements to applications We discuss the design and implementation issues in Utopia a load sharing facility speci cally built for large and heterogeneous systems The system has no restriction on the types of tasks that can be remotely exe cuted involves few application changes and no operating system change supports a high degree transparency for remote task execution and incurs low overhead The algorithms for managing resource load information and task placement take advantage of the clus tering nature of large scale distributed systems centralized algorithms are used within host clusters and directed graph algorithms are used among the clusters to make Utopia scalable to thousands of hosts Task placements in Utopia exploit the heterogeneous hosts and consider varying resource demands of the tasks A range of mechanisms for remote execution is available in Utopia that provides varying degrees of transparency and e ciency A number of applications have been developed for Utopia ranging from load sharing command interpreter to parallel and distributed applications to distributed batch facil ity For example an enhanced UNIX command interpreter allows arbitrary commands and user jobs to be executed remotely and a parallel make facility achieves speedups of or more by processing a collection of tasks in parallel on a number of hosts Such per formance is substantially better than that achieved using kernel based process migration in experimental operating systems such as Sprite For correspondence contact Songnian Zhou at CSRI University of Toronto King s College Road Toronto Ontario CANADA M S A Tel Email zhou white toronto edu Pierre Delisle is currently with Sun Microsystems Inc Boul Dr Fredrik Philips Bureau Saint Laurent QC Canada H M X